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Abstract 

We present a systematic small-correlation expansion to solve the inverse Ising problem: find 
a set of couplings and fields corresponding to a given set of correlations and magnetizations. 
Couplings are calculated up to the third order in the correlations for generic magnetizations, 
and to the seventh order in the case of zero magnetizations; in addition we show how to 
sum some useful classes of diagrams exactly. The resulting expansion outperforms existing 
algorithms on the Sherrington-Kirkpatrick spin-glass model. 

1 Introduction 

Calculating average values of observables given a Hamiltonian is a general problem in statistical 
mechanics. This can done either analytically for a few exactly solvable systems or numerically 
through simulations with e.g. Monte Carlo techniques. These techniques give access, for not too 
low temperatures or too big systems, to the local magnetizations rrii and spin-spin correlations 
Cij of an Ising sample, even in the notoriously complex case of spatially distributed interactions 
Jij and fields hi [1. Much less attention has been brought in the physics literature to the inverse 
problem, that is, calculating the couplings and fields from the knowledge of the magnetizations 
and correlations, a problem known as Boltzmann-machine learning in statistical inference theory 
[2] . Yet the growing availability of data in many biological systems of interest as neural assemblies 
[21 dj, proteins [5], gene networks [B], ... have strengthened the need for efficient techniques to infer 
interactions from correlations [7]. 

The purpose of this paper is to present a systematic expansion procedure to solve the inverse 
Ising problem. Given a set of observed magnetizations and correlations we look for the (a priori 
non uniform) couplings and fields of the Ising Hamiltonian reproducing those average observables 
at equilibrium. Our procedure is inspired from works by Plefka on mean- field spin glasses |10| . 
and subsequent results by Georges and Yedidia [TTl [T^] , who derived the free-energy of a spin-glass 
at fixed magnetization and interactions, performing a Legendre transform of the free-energy with 
respect to the fields. Technically speaking our work is an extension where one more Legendre 
transform, this time with respect to the interactions, is carried out to obtain the free-energy at 
fixed magnetization and correlations. 

The need for calculating free-energies under some constraints is not new. One well-known 
example comes from the physics of gas or liquids, where one looks for the free-energy of interacting 
particles at fixed density and pair correlations [8]. Another example can be found in field theory, 
where one is interested in determining the thermodynamic potential for fixed average values of 
the field and two-point correlations [S]. Calculations generally rely on expansions in powers of 
the correlations around the non-interacting case which can be exactly handled. It is important 
to stress that, in contradistinction with the above-mentioned examples and most of the existing 
literature, our work deals with the case of discrete spin variables and non-translationally invariant 
interactions. 
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The plan of the paper is as foUows. The general procedure for the expansion is exposed in 
Section [2l Section [3] is devoted to the generic case of non-zero magnetizations while Section |4] 
concentrates on the simpler case of zero magnetizations where the expansion can be pushed to 
higher orders. The results for the couplings are checked on two standard models: the unidimen- 
sional Ising model, and the Sherrington-Kirkpatrick (SK) model of a spin-glass. We show that our 
procedure for inferring couplings works better than existing methods for the SK model. The major 
technicalities are presented in the Appendices; the reader interested in explicit expressions for the 
couplings given the correlations and magnetizations can skip Section 

2 Procedure for the Small c Expansion 

We consider an Ising model over N spins (T; = ±1, i = 1, . . . , A^, with Hamiltonian 

H{{Si}) = - ^ JijCJiCTj ~ ^ hiUi . (1) 

i<j i 

We want to find the values of couplings and the fields, Jij,h* such that the average values of 
the spins and of the spin-spin correlations match the prescribed magnetizations and connected 
correlations Cy , 

"^. = ^^(U- , c.,^^^{{J:,},m)-m,m, (2) 

where the partition function (at unit temperature) reads 

Tr . (3) 

These couplings and fields are the ones that minimize the entropy of the Ising model at fixed 
magnetizations and correlationtQ, 

Si{J^,}Ah^};{m,}Ac^J}) = logZ({J,,},{/i,})-^J,,( Cij + mirrij) — ^ h^rrii (4) 

i<j i 

= log Tr exp I ^ J,y [(ct^ - mi){aj - nij) - Cy] -|- ^ Xi{ai - nii) ^ 
^""^ [i<i i J 

where the new fields A; are simply related to the physical fields hi through Xi — hi + Jijirij. 

The calculation of the entropy ^ for a given set of Jy and Xi is, in general, a computationally 
challenging task, not to say about its minimization. To obtain a tractable expression we multiply all 
(connected) correlations Cy in ([5]) by a small parameter /3, which can be interpreted as a fictitious 
inverse temperature. The calculation of the entropy S{{Jij}, {Xi}; {m^}, {/3 Cy }) is straightforward 
for P = since spins are uncoupled in this limit. The values of the couplings and fields minimizing 
the P — entropy are thus 

J*.(/3 = 0) = , A*(/3 = 0) = /i*(/3 = 0) = tanh"^(m,) . (5) 

Our goal is to expand the couplings and fields in powers of /?; to each order of the expansion the 
couplings and fields will be functions of the magnetizations and correlations. Ideally the couplings 
and fields we are looking for will be obtained when setting /5 = 1 in the expansion. 

To implement the expansion of J*j and A* from equation ([4]) we proceed in the following way. 
First we define a potential U over the spin configurations at inverse temperature /3 through 

... ^ ■ Jo 

^Note that the minimum may be reached for infinitely large values of hi or Jij i.e. as happens for fully correlated 
sites {aifTj) = 1. 



and a modified entropy, compare to ([4]), 



5({m.},{c.,},/3) = log Tr e^»--» 



(7) 



Notice that U depends on the coupling values J*j{/3') at all inverse temperatures /?' < (3. The true 
entropy (at its minimum) and the modified entropy are simply related to each other, 



5 = S* - ^ / 



d/3'j*(/3') 



(8) 



The modified entropy S ([7]) has an explicit dependence on (3 through the potential U and an 
implicit dependence through the couplings and the fields. As the latter are chosen to minimize S 
the full derivative of S with respect to /3 coincides with its partial derivative, and we get 



dS_ 

d/3 



1<J 



The above equality is true for any /3. Consequently S is constant, and equal to its [3 
that is, to the entropy of N uncoupled spins with known magnetizations {wi}. 
We now present three facts, shown in the Appendices: 



(9) 
value. 



A. For any integer fc > 2, 



dp 



fe-i 



Qk 



(10) 



where Qk is a (known) function of the magnetizations, correlations, and of the derivatives in 
(i — Q oi the couplings J,* and fields A* of order < max(l, k — 2). See Appendices 1^ and [Cl 
Recall that S is constant by virtue of ^ thus both sides of pU]) vanishes. 

B. For any integer k > 2 the /c*'' derivative of A* in /3 = can be calculated from the magneti- 
zations and the knowledge of the derivatives in (3 — oi the couplings J*j of order < k — 1. 
See Appendix [B] 

C. The first derivative of the couplings in /3 = is given by 



dJ* 



dp 



(11) 



See Appendix lA.il 



Those facts allow us to calculate the derivatives of the couplings in /3 = to any order in a recursive 
way. Let k > 3. From the definition ([4]) of the entropy 



dS 

dc,i 



{{J:AAK}■Am^},{pc.J}) = -|3J:, • 



(12) 



Differentiation of the above equation k times with respect to /3 in /3 = gives 



Using relationship 



AC 

^({4}'{^*};W,{/3c.,}) = 

^^"^j 



fik—1 



dp 



k-l 



we obtain 



_d 

dc, 



dp^ 



d^-^J: 



dp 



k-l 



OJ 



= -fc 



dp 



(13) 



(14) 



We now use that S is constant and fact A to deduce 



dQk 



(15) 



As a consequence the (fc - 1)*'' derivative of J*^ in /3 = is a known function of the derivatives 
in /3 = of the coupUngs J*j and fields A* of order < fc — 2 (and of the magnetizations and 
correlations). Using fact B we express all the derivatives of the fields in terms of the derivatives 
of the couplings of order < fc — 2. Hence we can compute the (fc — 1)*'' derivative of the couplings 
from the knowledge of all derivatives with lower orders. The recursive procedure uses fact C as a 
starting point to generate all derivatives. 



3 General results for non-zero magnetizations 

3.1 Explicit expansions of the entropy, couplings and fields 

The procedure exposed in the previous Section has allowed us to expand the entropy S and the 
fields hi up to order and the couplings up to order c^. Details are given in Appendix [XI We 
define 

(16) 



Li = I — rrii 



LiLj 



The entropy reads 

i 



1 + rrii , I + rrii \ — rrii , 1 — 
In H In 



2 ^KlL,Lj + -P^^Klm,mjLiLj+ (3^ ^ K,jKJkKk^L,LJLk 

i<3 i<j i<j<k 

- ^ E [1 + 3"^^ + 3"^' + 9"!- L,L, " y E E KIK^LIL^L, 

i<j i<j fc 

- Z?'' ^ {KijKjkKkiKu + KikKkjKijK^i + KijKjiKikKki)LiLjLkLi 

i<j<k<l 



(17) 



The terms in the expansion can be represented diagrammatically. A point in a diagram represents 
a spin, and a line represents a Kij link. We do not represent the polynomial in the variables {nii} 
that multiplies each diagram. Summation over the indices is implicit. 



S{{ckl},{m^},f3) = -•- 



1 




(18) 



In contradistinction with [TI] the expansion includes non-irreducible diagrams. It should be noted 
that, as in [TT], the Feynman rules of these graphs is unknown even in the rrii = case, which 
makes impossible to do the expansion by a simple enumeration of the diagrams. The result for Jy 
is 

J* {{cki}, {m J, P) = f3K,^ - 2l3^m,m^Kl - ^i^Kk^L^ 

k 

+ ^p'Kf^[l + 3mj + 3m] + 9mlni]]+p^ ^ K,,{K%L, + KlL,)Lk 



0" J2 KjkKkiKuLkLi + 0(13^) 



(19) 



k,l 



We can also represent Jj* diagrammatically, with the difference that we connect the i and j sites 
with a dashed Une that do not represent any term in the expansion: 



T* _ 



- 2< 




(20) 



We end up with the expansion for the 'physical' field 



In 



1 - mi 



2[3^mi ^ ^ K^kKkjKjiKuLiLjL 



i<j k 

j 



am: 



(21) 



The diagrammatic representation of hi is very similar to the one of 5 (not shown). 

We have tested the behaviour of the series on the Sherrington-Kirkpatrick model in the para- 
magnetic phase [13] • We randomly draw a set of iV x (TV — l)/2 couplings JlJ"*^ from uncorrelated 
normal distributions of variance J'^/N, calculate the correlations and magnetizations from Monte 
Carlo simulations, infer the couplings J*j from the above expansion formulas and compare the 
outcome to the true couplings through the estimator 



A = 



N{N - 1)J2 



jtrue 
■-'ij 



(22) 



i<j 



The quality of inference can be seen in Figure [T] for orders (powers of /3) 1,2, and 3. For large 
couplings the inference gets worse when the order of the expansion increases, as could be guessed 
from the presence of terms with alternating signs in the expansion, compare the 2-site loop, triangle, 
and square in (dH), (^0)) . 



3.2 Resummation of loop diagrams 

The divergence coming from the alternate series can be cured by summing all loop diagrams. A 
simple inspection shows that each diagram is multiplied by ±1 depending on the parity of the 
number of its links. From an algebraic point of view 



^(loop) 



?2 



KjkK^Lk + y KjkKkiKuLkLi + 



k.l 



= (L.L,)-'/'[M-(Id + M)-i)]^, 



(23) 



where M is the matrix defined by = fiKij yjLiLj and M,;i — 0. Expression ([23)) for the 
coupling was already known as a consequence of the TAP equations (see jT3] and [T^), and is 
exact up to 0{1/N) corrections for infinite range models. Our calculation shows how models with 
0(1) couplings depart from the TAP expression, 



J*i{{cki},{mt}) = Ji 



*(loop) 



2(3^m,mjKl + -p^^K^ [-1 + Sm^ + 3™^ + 3m>2] + 0(^4^ ^24) 
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Figure 1: Relative error A (|22p on the inferred couplings as a function of the inverse temperature 
J of the Sherrington-Kirkpatrick model with N — 200 spins. Monte Carlo simulations are run over 
100 steps, and averages and error bars are computed from 100 samples. Top: orders 1,2, 3 of the 
expansion. Bottom: expression (|24p which includes the sum over all loop diagrams. Inset: largest 
eigenvalue A of matrix M as a function of J . 



Figure [T] shows how the resummation of loop diagrams eliminates the divergence in the relative 
error A as expected. The same phenomenon takes place in the simpler Curie- Weiss model of a 
ferromagnet where spins interact through uniform couplings Jy = Jq/N, and the (connected) 
correlations are of the same order, Cy = c/N. From the relation Nc — dm/dh we can deduce that 
the large- iV expression for the coupling 

J, = -^ = c-c'+c^-c^ + ... (25) 
1 + c 

is an alternating series with radius of convergence c = 1. This radius is also given by the condition 
that the largest eigenvalue of equals l|j. This condition applies to the general case too: a 
necessary condition for the convergence of equation is that the largest eigenvalue A of M must 
be smaller than unity. We plot in the Inset of Figure [1] the behavior of A as a function of J. It 
appears that A = 1 for J ~ .3, a value comparable to the intersection point of the lowest order 
expansions, J ~ .35. 

The apparent large value of the relative error A in Figure [T] is not due to the quality of the 
expansion but to the the noise in the correlations and magnetizations introduced by the imperfect 
sampling of MC simulations. We show in Figure [Dhow the absolute error J x A decays as the 
square root of the number of MC steps, and is roughly independent of J (except close to the 
spin-glass temperature J — 1). As expected, for an infinite number of MC steps and N oo, the 
error should vanish. 



■^The on-diagonal entries of the correlations are chosen to be -as is the case for diagonal couplings— while 
off-diagonal coefficients coincide with aj. 
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Figure 2: Absolute error J x A on the inferred couplings as a function of the inverse temperature 
J of the Sherrington-Kirkpatrick model. Inference is done through formula (|24|) , which takes into 
account all loop diagrams. The error decreases with the number of spins and the number of Monte 
Carlo steps (shown on the figure). 



3.3 Resummation of two-spin diagrams 

Looking carefully at the results of the Section 13.11 one can deduce a general formula for the two 
spins diagrams, 

^*(2-spin) ^ _ 2m,mjKf^ + ^Kf^{l + 3m^^){l + 3m^^) - AKf^m.mjil + mf){l + w?^ 

k -1 ^ 1 - mf I - mj 

= ^ In [1 + Kij{l + m,){l + nij)] + i In [1 + K,j{l - m,)(l - m^)] 

- iln[l-if,,(l-m,)(l + mj)]-iln[l-X„-(l+m,)(l-mj)] (26) 

Where we have used equation ([75)1 from Appendix [C] to evaluate the averages. This expression 
is exact, and was checked by a symbolic calculation program. Note that in the case of zero 
magnetization, (j26p simplifies to J*^"^ s^pms) _ ^^^^j-^-i f,^.^ 

The resummation of all 2-spin diagrams and loop diagrams can be done, with the result 

T*(2-spin+loop) _ T*(loop) T*(2-spin) Kjj 

tj J 

The last term in (j27p prevents double-counting of diagrams of the type ■c:> , (obtained 

through contraction of [___]), and is derived in Appendix IdI The compact expression (1271) contains 

all the diagrams present in (|19p. in addition to higher order loop and 2-spin contributions. 

Resummation of all diagrams with a larger number k of spins is harder. It is done in Section 
14.21 in the case of zero magnetizations and fc = 3. For larger values of k we are not aware of any 
closed analytical expression, and resummation can be done by means of numerical procedures only. 
An important remark is that contributions from diagrams with k spins behave as 0(J^j^2(l 
when the m^s tend to 1 (or -1) as we show in Appendix [C] This expansion is particularly adapted 
to the inference of couplings from strongly magnetized data; a practical application can be found 
in [H. 



4 Further results in the zero magnetization case 



4.1 Higher order expansions of the entropy, couphngs, and fields 

While the procedure described in Section [2] allows for a systematic expansion of the couplings 
in powers of (3 it is technically involved to do by hand. In this section we find numerically the 
expansion up to order 0{(3^) in the simpler case where = for all spins i. 

We know that the expansion of S up to fifth order is given by the sum of all diagrams with 5 
links or less. More precisely, 




As we already know S^^^"^ ""^^"^ from the previous Section what remains to be found are the 
coefficients. According to the procedure outlined in Section [2] those coefficients are rational (and in 
particular, for low orders, with a small integer denominator). Our idea is to find those coefficients 
from a fit of a numerical solution. 

Numerically we minimize the entropy (jj]) for a small number N of spins (not larger than 
eight). Correlations are arbitrary numbers chosen to be very small (about 10~^) since we want 
the corrections of the order of O(c^) to be numerically negligible compared to O(c^) terms. Of 
course, when the correlations are very small, so are the inferred couplings. To estimate the latter to 
sufficient accuracy we have performed our calculations with a unusual large number of decimal units 
(w 400). A computer program, at each step /, randomly chooses the couplings and numerically 
evaluates the corresponding entropy Si and correlations c\j through an exact enumeration over the 
2^ spin configurations. Then it calculates 



D = 



1^ 

n 

1=1 



_^(5th order) 



(29) 



over a large number L of random samples. This quantity is quadratic in the coefficients a^, so its 
minimum can be easily obtained, and we could deduce that the coefficients in the expansion (|28p 
are all zero. Using this procedure, order by order, we have determined the following expansion for 
J*j (where the coefficients found numerically differed from the rational fractions listed below by 
less than 10~^°): 



T* (2-spin+loop) 




(30) 



We can note the absence of any term with five or more spins in this expansion. We suspect 
that the lowest order diagram in this expansion for a given number spins is the loop with double 

links. In particular, the first diagram with five spins would be , which is not present in ([50)1 

since it is O(c^). 



4.2 Three spins summation for zero magnetization 

For three spins and zero magnetizations the entropy (jlj can be minimized exactly with a symbohc 
algebra program, with the following results for the couplings 

■ 1 + a 



J, 



-(3-spin) 



- E 



log 



1 



Cik + Cjk 



log 



1 - C,„- + Cjfe - Cjk 



1 



^ik ~t~ ^jk 



log 



1 + Cy + Cik + Cjfe 



1 



Cik + Cjk 



(31) 



Following the same lines as in Section 13.31 we gather our previous results in the = case under 
the forno, 



* (2-spin+loop+3-spi] 



y-*(2-spin+loop) , y-*(3-spin) 



*(2 spins) 



(32) 



4.3 Check on the one-dimensional Ising model 

We consider a unidimensional Ising model with uniform coupling J between nearest neighbour^. 
We have 

c,y = |tanh J|l*"^'l (33) 



J, 



*(2-spin) 



tanh ^ ( |tanh J| 



(34) 



We can see from the above formula that the sum of all 2-spin diagrams infer the correct value of 
the couplings Jis+i, but give a non-zero value for the other ones. We can also evaluate the sum of 
loop diagrams (with c = tanh J): 



J. 



<loop) 



+ 5is-i) ■ 



, o I "i.i— i; ) (35) 

1 — 

where is the Kronecker function. We obtain a zero contribution for non-neighbouring sites, 
but an erroneous values for the nearest-neighbour coupling J^^i+i. If we consider the contributions 
from both 2-spin and loop diagrams, 



J, 



(2-spin4-loop) 



J(^i,i+i + 5i^i-i) + 



tanh Ci 



1 - c2. 



(1 - (5i,j+i)(l - (5i,i_i) 



= J(<5m+i+^^,.-i) + 0(c^) , (36) 
which is correct to the order c^. The next contribution to the couplings coming from the expan- 
sion (j30[) corresponds to ^i-i^ , whose leading term is indeed proportional to • cf ■ cf^i j_|_2 = 



We may also want to understand how Jis+2 converges to zero. Using geometric series calcula- 
tions one can evaluate 



(37) 




[l-c^Y 



(38) 



Performing the whole summation in (j30p we find that = O(c^), which is consistent with the 

first missing contribution from the expansion. 



■^Calculations to avoid double-counting are similar to the ones shown in Appendix [D] with a 3 X 3 instead of 2 X 2 
matrix, and are not shown. 

*The case of non-uniform couplings varying from link to link can be treated along the same lines. 



4.4 Application to the Sherrington-Kirkpatrick model 



We have seen above that the error on the inferred couphngs for the Sherrington-Kirpatrick model 
is essentially due to the noise in the MC estimates of the correlations and magnetizations. To avoid 
this source of noise we now evaluate the error due to our truncated expansion using a program that 
calculates Cij through an exact enumeration of all 2^ spin configurations. We are limited to small 
values of N (10, 15 and 20). However the case of a small number of spins is particularly interesting 
because, for the SK model, the summation of loop diagrams is exact in the limit N ^ oo. The 
importance of terms in our expansions not included in the loop resummation is thus better studied 
at small N. 

Results are shown in Figure [31 The error is remarkably small for weak couplings, and get dom- 
inated by finite-digit accuracy (10"^'^) in this limit. Not surprisingly it behaves better than simple 
loop resummation, and also outperforms the message-passing-based method recently introduced in 



0.01 
le-04 
le-06 

A 

le-08 
le-10 
le-12 



le-14 

0.01 J 0.1 1 

Figure 3: Relative error A as a function of J for the SK model for our resummation 

j{2 spin -I- loop + 3 spin) Compared to the Susceptibility Propagation method of Mezard and 

Mora [7] and loop resummation ([24]). 

5 Perspectives 

As we saw in Sections 14.31 and 14.41 the expansion method introduced in this paper works well 
for both the Sherrington-Kirkpatrick sping-glass -an infinite dimensional system, with very dilute 
couplings- and the unidimensional Ising -with only a few but strong couplings per site- models. It 
would be interesting to investigate how accurate our method is for 'Small- World'-like interaction 
networks, which have both kinds of couplings [TB] . 

In principle the assumption of binary- valued spins (ci — ±1) is not central to our expansion 
and could be straightforwardly released to tackle the case of Potts models, where each spin can be 
in q possible states (ct^ = 1, ...,q). Such a generalization would make the method useful to connect 
with biological problems involving amino-acids [5]. 

Finally our expansion breaks down with the onset of the spin-glass phase as can be seen from 
Figure [3] The failure of our method (and of other existing algorithms) is not surprising. Correla- 
tions and magnetizations have a physical meaning when there is a single pure state. In presence 
of more than one phases Gibbs averages have indirect significance. A well-known example is the 
ferromagnet at low temperature and zero field where two equally-likely phases of opposite magne- 




tizations im^ exist, and the resulting Gibbs magnetization m truly vanishes (for any finite N). 
Work is in progress to extend our expansion technique to this multiple-phase regime. 
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A Details of the small-/? expansion 

Let O be an observable of the spin configuration (which can explicitly depend on the inverse 
temperature f3), and 

(0) = iTrOe^ (39) 

its average value, where U is defined in ([6]), and Z = cxp(5). The derivative of the average value 
of O fulfills the following identity. 



d{0) _ 1 



Tr 



do dU_ 
'0(3^ 'dj3 



dp Z {<T.} 

where the term in Z^^ vanishes as a consequence of ^ 



O 



dU_ 
'df3 



(40) 



A.l First order expansion 

Using (gOl) and dH), 



a/32 gp \ gp 
and by using the explicit form of U given in 



9^2 



d(3^ 



d T* 

" ^ d(3 



dU_ 

a/3 



(l-n.D(l-m|)+E(9- 



d\* 



(41) 



(1 - mj) (42) 



In Appendix[B]we show that 



= 0. As 5 is constant we end up with the following algebraic 



equation for the first order derivative of J*. (/3) , 



dJ* 



d(3 



(1-to2)(1_™2) _ 



(43) 



The only non-zero solution of the above equation, symmetric under index permutations, is the 
announced result (fTTjl . 

A. 2 Second order expansion 

Using (go]) and gT]) 







d^S _ d 
'dp^~dp 



9/32 



dU_ 
'dp 



/dHJ_\ /d^UdU 



dU_ 
'dp 



(44) 



A straightforward calculation gives (where we omit for clarity the notation |o and the * subscript 
from Jij and A;) 



d^UdU\ 
a/32 gp/^ 



dU 
d(3 



'2 



i<j 



9/?2 dp 

dJij dJjk dJk 



9/32 dp ' 



(45) 
(46) 



6 ^ 



i<j<k 



dp dp dp 



dp dp dp ' ' 



(47) 



Using (UH), results from Appendix [Bl for the expressions of the derivatives of in /? = 0, and 
pT]) we obtain JTU]) for fc = 3 with 



Q2 = -4E7I 



(l-™?)'(l-m2)2 " (l-m2)(l-m2)(l-m2) 



from which we deduce 
and 



= 4^ JCfj-TOimjLiLj + 6 ^ Ki^K-jkKkiLiLjLk 



i<3 



i<j<k 



52j,v 



5/32 



= -AmiirijKfj - 2 ^ KjkKkiLk 



(48) 



(49) 



(50) 



A. 3 Third order expansion 

The procedure to derive the third order expansion for the couphng is identical to the second order 
one. We start from 



= 



d^S /d*U 



dp* \ dp' 



a/52 j J+ \ gp3 gp 



duV d^u' 



dp J dp^ 



and evaluate each term in the sum: 



/dHJ_dU\ 



dp J 5/32 



dU 



dp 



a/33 



E 



9/32 



KijLiLj 



(9/32 



n 2 



i<3 



a/33 



(52) 

(53) 
(54) 



d^Jk' 52 J.. 

2 E E ^ij^Jk-g^LiLjLk + 4 ^ Kl-^rmmjULj 

i<k j 



52 A 



9/32 



(55) 



«<j 



= 5^i^,^.(3m? + l)L,(3m| + l)i,- + 3 ^ KfjKl.LiLjLkLi + 
+ 6 ^ ^ ^K],{3m'. + l)LiLjLk + 



i<k j 



+ 12 ^ KijKjkKkiLiLjLk [AmirrijKij + AmiirikKik + AmkiriiKki] + 

i<j<k 

+ 3 ^ KijKjkKkiKuLiLjLkLi 

hJ,k,l(^) 



(56) 



Using the results from Appendix B we can write all the terms above in the same form 

2 



-3 



i<], k<l (k^i, l^j) 



K?,KliLiLjLkLi 



Kj k 



i<j k i<j 

-"^S^^KfjKikKkjm^mjLiLjLk - 

i<j k 

12 ^ KijKjkKkiKuLiLjLkLi 

^,j.kd(^) 



(57) 



(58) 



Kj k 



E^= 



i<j k 



3E 



92 J, ^ ' 



a/32 



i<j i<j k 

+ 6 ^ KijKjkKkiKuLiLjLkLi 

^,3,k,l (#) 

+ 12^^i^i.i^2^.L2^,L, 



(60) 



3E 



i<j k 



i<j k 



(61) 
(62) 



i<j k 



Again we find equation ^TU\i with 

Q3 = - E [(3"^' + 1)(3?"' + 1) - 48m2m2] L^Lj 

i<j 

i<3 k ij,k,l {=^) 

+ i2EE^^™'^^^' + 3E^^'^' 

i j i<j 

which gives the fourth order contribution to the entropy, 

S = -2 E ^t, [1 + + 3m2 + 9m2m2] L,L, - 12 E E ^I^IA^^^^ 

i<j i<j k 

- 24 ^ {K^JKJkKklKu + K,kKkJKlJKil+K,jKJlKlkKk^)L,LJLkLl 

i<j<k<l 

and the third order contribution to the coupling, 



(63) 



(64) 



+ 6 ^ KjkKkiKuLkLi. (65) 



k I 



B Derivatives of A* in (3 = 

Since rrii and hi are conjugated thermodynamic variables it is natural to evaluate 
dS I dU 



dm I 



dm I 



dp' 1'' ' 
dmk 



dJ* 



i<j 

-X 



' d^ W 

dmk 



(66) 



As the modified entropy is independent of /3, 
dS dS 



dmk dmk 



= -Xm = tanh-\mk) = I In f}±I!^) 
Q 2 \l-mkj 



(67) 



where we used the well-known result for the entropy of uncorrelated spins. We can then deduce 
the formula, valid for any f3: 



2 \l + mk 



i<j 



d/3' 



dmk 



(68) 



It is now straightforward to deduce the expansion of A* to the order 0{P^) from the expansion of 
Jj* to the order 0{P^^^). In particular, 



d(3 



and using the order 0{(3) of the J*j expansion. 



5/32 



= 2mkY,KlL, 



(69) 



(70) 



C Large magnetization expansion 

Equation suggests that to expand J*^ to the order of {Li)^ one has to sum all the diagrams 
with up to fc + 2 spins. This statement is true if the expansion for J*^ is of the form 

J* = Ay + X LfeAy-fc + X! E LkLiA.jki + ••• (71) 

k k I 

where the coefficients Aiji2...j„ Eire polynomials in the couplings Ki^i^^ and the magnetizations m„ 
{a, (3 < n). In the following we will show that the above statement is true to any order of the 
expansion in /3 by recurrence. First of all, from 168]) we see that if J*j is of the form ([TTjl up to the 
order k, so is A* to the same order. 

As we saw in section [2l to find an equation for , one must evaluate g^k+^i ■ Using Eq. [40l 
we can write 



Qk+lg 




(72) 

where a is a multi- index and \a\ = fc+ 1, and Pa a multiplicity coefficient. The highest order term 
of this expression evaluates to J2ij LiLjKij ^^'^y — 



Due to the structure of U, spin dependence in (|72p will come either from the lower derivatives 
of Jj* (of the form (|7ip by hypothesis), from the derivatives of A*, or explicitly from U. In the 
later case we get a multiplicative factor (tJi — rrii). Hence we end up with computing a term, with 
fc > 1 , of the form 

- m..r) ^ i-im - (73) 

Clearly any term including (cr,; — mi) will give a multiplicative factor Li after averaging. As spins 
are decoupled in the /? = limit we obtain the product of those factors over the spins in the 
diagram as claimed. 

D Double-counting 

We want to remove two-spin diagrams from the resummation of loop diagrams. These two-spin 
diagrams are precisely the ones appearing in the loop diagrams in a system including two spins 
only. In the case of iV = 2 spins the matrix M reads 



(74) 



*'(loop) 
3 

the contribution to be subtracted to the sum of 2-spin and loop diagrams (third term in (j27[) ). 



We then calculate J*^ for this simple N — 2 spin model from formula ([^5]) , and get this way 
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